1. Field of the Invention
The present invention relates to systems for assisting a driver of a vehicle to park via an image of around the vehicle and, more specifically, to a system for calculating a distance from the vehicle to a specific 3D object in the image picked up from the vehicle and advising the driver of what is located around the vehicle.
2. Description of the Background Art
Conventional technologies so far disclosed as relevant to vehicles, such as automobiles, include detecting any obstacle around a vehicle, for example. If any obstacle is detected, and if a collision with the obstacle is determined as being highly possible, a driver of the vehicle is warned by alarm, advised of a distance to the obstacle, or according to the distance, the vehicle is automatically braked to stop. As an example of such detection means, there is a distance detection system utilizing an image recognition technology, such as disclosed in Japanese Patent Laid-Open No. 5-114099 (93-114099). Utilized in this system is an image picked up by a camera mounted in a vehicle.
The distance detection system is now described with reference to FIGS. 11 and 12. Specifically, FIG. 11 is a diagram showing the relationship between a camera and a 3D object, while FIG. 12 a block diagram showing the distance detection system.
In the distance detection system, a vehicle, such as in an automobile, is equipped with a stereo optical system for picking up an image of the 3D object located outside of the vehicle within a given range. The stereo optical system is exemplarily implemented by several cameras utilizing a solid image-pickup device, such as a charge-coupled device (CCD), and each fixedly attached to a ceiling surface of the vehicle towards its front. Here, the number of cameras is presumably two, and the camera located on the right, as viewed from the front, is referred to as the right camera, and the other camera on the left is as the left camera.
In FIG. 11, the right and left cameras are placed so as to be away from each other by a distance s, and each of the cameras photograph a point P at a perpendicular distance D therefrom. Assuming a focal distance for both cameras being f, an image of the point P is picked up on each camera""s projection surface, which is away from the camera by the focal distance f. In this case, the distance between these two images on the projection surfaces is s+t, where t denotes a displacement. Thus, the distance D is so calculated as follows:
D=s*f/t
That is, the displacement t leads to the distance D.
Described next is the process of calculating the displacement t. In FIG. 12, a stereo optical system 100 forwards data two of analog images picked up therein to an image processing part 201. The two analog image data are then subjected to an A/D conversion for output to a hamming distance calculation part 202. The two digital image data are then calculated, respectively, for a hamming distance H on a pixel basis. The hamming distances H are used to determine the displacement of the two images. A minimum/maximum value detection part 203 then searches the hamming distances H calculated by the hamming distance calculation part 202 for a minimum value Hmin and a maximum value Hmax. It is herein assumed that the minimum value Hmin is the displacement t.
As such, the distance detection system 200 processes two images picked up by a stereo camera mounted in a vehicle, and finds the displacement t. Thereby, the system can assist a driver with a 3D position of an obstacle, for example, and a warning alarm is possible to prevent against a collision or bump.
Such a distance detection system, however, requires considerable accuracy for fixedly attaching several cameras of the stereo optical system to a vehicle, especially with regard to angle, direction, interval, and the like. There also needs to be previously input, to a stereo image processing device, data indicating the satisfactory positions of those cameras. As a result, structuring such a stereo optical system is very difficult. Further, as described in the foregoing, in order to calculate the hamming distances H, object detection needs to be performed for one image with reference to the other image in their entirety. Also, the search on the pixel basis needs to be repeated, the amount of calculation is thus quite large, and the hardware structure is complicated. Still further, the distance to an obstacle advised by this distance detection system is often in numerical figures or by images being quite different in appearance from the actual obstacle. Therefore, the driver cannot quite perceive the actual distance.
Therefore, an object of the present invention is to provide a parking assistance system for assuredly advising a driver of a distance between his/her vehicle and an obstacle in a perceivable manner, while easing attachment of an image pick-up to the vehicle and reducing the number of calculations required for image processing.
The present invention has the following features to attain the above object.
An aspect of the present invention is directed to a parking assistance system for assisting a driver of a vehicle to park by picking up an image around the vehicle, and advising the driver of a 3D object located therearound. The system comprises an image pick-up for picking up, with a single camera during the vehicle""s transition, first and second images around the vehicle picked up at a different location and first and second times, an object specification part for detecting and specifying the 3D object found in each of the first and second images, a vehicle position calculation part for calculating transition data of the vehicle from the first location to the second location, an object distance calculation part for calculating a distance from the vehicle to the 3D object by using the position of the 3D object in each of the first and second images, and the transition data of the vehicle, and an image creation part for creating a third image for display to the driver based on the images picked up by the image pick-up and data calculated by the object distance calculation part.
As described above, in the aspect of the present invention, as is provided with a single image pick-up, the parking assistance system picks up two images at different location and time, and calculates the transition data of the vehicle between the two locations. Therefore, with the principle of triangulation applied to the 3D objects in two images, the parking assistance system can correctly calculate the distance to the 3D object. Accordingly, one image pick-up will do, and the parking assistance system achieves easy attachment thereof as having no more need to fixedly attaching several image pick-ups with considerable accuracy, especially with regard to angle, direction, interval, and the like. Further, the data calculated by the object distance calculation part correctly indicates the distance from the vehicle, and thus the third image will not be askew and tell the correct distance. Still further, by displaying the image onto which the distance data is added, the actual distance from the vehicle to the 3D object is easier for the driver.
The object distance calculation part preferably detects edges of each of the 3D objects found in the first and second images. Also, the object distance calculation part uses the coordinate data of any edge identical in the first and second images and the transition data of the vehicle so that the distance from the vehicle to the edge of the 3D object is calculated. Therefore, even if the vehicle moves in a complex fashion, the distance to the 3D object can be easily calculated with only the coordinates data of any edge identical in the two images and the transition data of the vehicle. Further, in order to detect any identical edge, by using the transition data of the vehicle, the object distance calculation part may estimate the position of the edge identical in the first and second images, and then positionally subjects the edge to image analysis. In this case, since the transition data of the vehicle has been previously calculated, the positional relationship between the two images is clarified. This makes possible to estimate which edge in one image is located where in the other image, and image analysis is only done for the estimated part of the image, reducing the amount of calculation used for the edge detection. Moreover, the hardware structure can be simplified.
The image creation part preferably converts the image picked up by the image pick-up into a view from the above, and then creates a third image by adding the data calculated by the object distance calculation part onto the view. With the help of such created third image on the display, the driver can grasp the actual distance from the vehicle to the 3D object. Alternatively, the image creation part may perceivably emphasize, out of the data calculated by the object distance calculation part, a part of the object in the vicinity of the vehicle. Thereby, out of the data calculated by the object distance calculation part, a part of the object in the vicinity of the vehicle may be emphasized by color, line width, or the like, to let it stand out to warn the driver against approaching 3D object. The image creation part also may generate the third image by converting the data calculated by the object distance calculation part into a 3D object model, and adding the 3D object model onto the view from the above. As such, by adding the close-to-actual 3D object model onto the view from the above, the driver is warned about the 3D object, and it also becomes easier to perceive the distance between the 3D object and the vehicle. Or, the image creation part may additionally add a model of the vehicle onto the third image. As it is equipped in the vehicle, the image pick-up cannot pick up an image of the vehicle in its entirety. In this sense, the image is not complete, and thus the driver may have a hard time grasping the actual distance from the vehicle to the 3D object. To get around such a problem, the model of the vehicle is fixedly displayed on the third image.
The object specification part, as an example, generates a fourth image by projecting the first image onto the second location for transformation, and then takes a difference between the fourth and second images to detect and specify the 3D object. Alternatively, the object specification part may generate the fourth image by projecting the second image onto the first location for transformation, and then takes a difference between the fourth and first images to detect and specify the 3D object. Either structure will do to detect and specify the 3D object in the image in an easy manner. This is because, due to the extraction of such differences between one image actually picked up and the other obtained by projecting the former image thereonto, only any askew part is left behind.
The transition data of the vehicle typically includes the linear transition distance and the vehicle angle of the vehicle. In such a case, based on data obtained from a steering angle sensor and wheel speed sensor via a pulse counter equipped in the vehicle, the vehicle position calculation part calculates the transit distance and the vehicle angle.
The image pick-up is preferably equipped at a rear part of the vehicle having many blind spots, especially when parking, thereby rendering the parking assistance system more useful. This is especially applicable to a large-sized vehicle.
In a preferable example, the above-described third image is displayed on the display for the driver. Since the third image looks quite close to the actual image, it allows the driver to easily perceive the actual distance.
Further, in another preferable example, when the distance from the vehicle to the 3D object calculated by the object distance calculation part is shorter than a predetermined distance, a sound and/or light is generated. Thereby, the driver can be warned against the approaching 3D object.
The image pick-up may be conditionally started in various manners. First, the system may start picking-up an image in response to the manual operation of the driver. If this is the case, the driver can selectively start the parking assistance system only when he/she needs its assistance. Second, the system starts when the vehicle decelerates to a predetermined speed. Third, the system starts when the vehicle goes in reverse. According to the second and third conditions, the vehicle can tell whether its in a parking mode or not. Thus, the parking assistance system can be started only when the vehicle is in the parking mode.
These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.